A bias correction function for classification performance assessment in two-class imbalanced problems

This paper introduces a framework that allows to mitigate the impact of class imbalance on most scalar performance measures when used to evaluate the behavior of classifiers. Formally, a correction function is defined with the aim of highlighting those classification results that present moderately higher prediction rates on the minority class. Besides, this function punishes those scenarios that are biased towards the majority class, but also those that are strongly biased to favor the minority class. This strategy assumes a typical imbalance task, in which the minority class contains the most relevant samples to the research purposes. A novel experimental framework is designed to show the advantages of our approach when compared to the standard use of well-established measures, demonstrating its consistency and validity. Most of traditional learning methods assume that the classes of the problem share similar prior probabilities and/or misclassifi-cation costs. However, in many real-world tasks the ratios of prior probabilities between classes are significantly skewed. This situation is typically known as the imbalance problem. A two-class data set is said to be imbalanced when one of the classes (the minority one) is heavily under-represented regarding the other class (the majority one) [1]. Paradoxically, the minority class is often the most important and usually the one with the highest misclassifi-cation costs. Some typical real-life applications where this problem arises are prediction of microarray gene expression [2], prediction of corporate bankruptcy [3] and credit risk [4], fraud detection in mobile telephone communications [5] and text categorization [6]. Because of examples of the minority and majority classes usually represent the presence and absence of rare cases respectively, they are also referred to as positive and negative examples. As pointed out by many authors [7–10], the use of plain accuracy and/or error rates to evaluate the performance of classifiers in imbal-anced domains might produce misleading conclusions, since they do not take misclassification costs into account, are strongly biased to favor the majority class, and are non-sensitive to class skews. A plethora of alternative scalar and graphical methods have been proposed to properly assess classification performance on imbalanced scenarios. Graphical approaches depict trade-offs between two or more evaluation perspectives, allowing a richer analysis of results but making the comparison of learning algorithms a non-trivial issue. Some well-studied examples are the Receiver Operating Characteristic (ROC) curve [11,12], the Precision–Recall (P–R) curve [13], cost curves [14] and the Bayesian Receiver Operating Characteristic (B-ROC) curve …